<html>
<head><meta charset="utf-8"><title>adding State type to the AllocRef API · t-libs/wg-allocators · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/index.html">t-libs/wg-allocators</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html">adding State type to the AllocRef API</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="217567013"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567013" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567013">(Nov 22 2020 at 19:56)</a>:</h4>
<p>Hey gang, I realize that it might be too late to propose changes to the allocator API (I thought that the proposals were dead and I didn't quite keep up to date with the progress).<br>
I think that it might be a good idea to have the AllocRef trait take a generic argument like "State" or "Id".</p>
<p>Why? Most GPU APIs when allocating buffer, don't return a pointer to bytes but some sort of object that can be used to get the actual memory buffer.</p>
<p>The problem with the current AllocRef design is that the allocator would have to keep track of the mapping between the pointers and the objects returned from the API so that .</p>
<p>The State type would allow the allocator to easily lookup which object it's dealing with. So all the allocator methods would now take an extra parameter called state.</p>
<p>I guess the functions would also have to be marked as mutable. </p>
<p>Let's say that you are writing a wgpu allocator, it would then look something like this</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">struct</span> <span class="nc">BufferId</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">inner</span>: <span class="kt">usize</span><span class="p">,</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">struct</span> <span class="nc">WGPUAllocator</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">device</span>: <span class="nc">wgpu</span>::<span class="n">Device</span><span class="p">,</span><span class="w"></span>
<span class="w">    </span><span class="n">inner</span>: <span class="nb">Vec</span><span class="o">&lt;</span><span class="n">wgpu</span>::<span class="n">Buffer</span><span class="o">&gt;</span><span class="p">,</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">impl</span><span class="w"> </span><span class="n">AllocRef</span><span class="o">&lt;</span><span class="n">BufferId</span><span class="o">&gt;</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">WGPUAllocator</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">fn</span> <span class="nf">alloc</span><span class="p">(</span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">layout</span>: <span class="nc">Layout</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nb">Result</span><span class="o">&lt;</span><span class="p">(</span><span class="n">BufferId</span><span class="p">,</span><span class="w"> </span><span class="n">NonNull</span><span class="o">&lt;</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="p">),</span><span class="w"> </span><span class="n">AllocError</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">buffer</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">device</span><span class="p">.</span><span class="n">create_buffer</span><span class="p">(</span><span class="o">..</span><span class="p">.);</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">id</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">BufferId</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">inner</span>: <span class="nc">self</span><span class="p">.</span><span class="n">inner</span><span class="p">.</span><span class="n">len</span><span class="p">()</span><span class="w"> </span><span class="p">};</span><span class="w"></span>
<span class="w">        </span><span class="bp">self</span><span class="p">.</span><span class="n">inner</span><span class="p">.</span><span class="n">push</span><span class="p">(</span><span class="n">buffer</span><span class="p">);</span><span class="w"></span>
<span class="w">        </span><span class="nb">Ok</span><span class="p">(</span><span class="n">id</span><span class="p">,</span><span class="w"> </span><span class="n">buffer</span><span class="p">.</span><span class="n">raw_memory</span><span class="p">())</span><span class="w"></span>

<span class="w">    </span><span class="p">}</span><span class="w"></span>

<span class="w">    </span><span class="k">unsafe</span><span class="w"> </span><span class="k">fn</span> <span class="nf">dealloc</span><span class="p">(</span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">,</span><span class="w">  </span><span class="n">id</span>: <span class="nc">BufferId</span><span class="p">,</span><span class="w"> </span><span class="n">ptr</span>: <span class="nc">NonNull</span><span class="o">&lt;</span><span class="kt">u8</span><span class="o">&gt;</span><span class="p">,</span><span class="w"> </span><span class="n">layout</span>: <span class="nc">Layout</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="bp">self</span><span class="p">.</span><span class="n">device</span><span class="p">.</span><span class="n">dealloc</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">inner</span><span class="p">[</span><span class="n">id</span><span class="p">]);</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>



<a name="217567178"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567178" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567178">(Nov 22 2020 at 20:01)</a>:</h4>
<p>how would, say, Vec&lt;T, A&gt; use and keep track of this state? or would it have to be <code>Vec&lt;T, A: AllocRef&lt;State = ()&gt;&gt;</code> or something?</p>



<a name="217567329"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567329" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567329">(Nov 22 2020 at 20:04)</a>:</h4>
<p><span class="user-mention" data-user-id="209168">@Thom Chiovoloni</span>  the latter i guess?</p>



<a name="217567366"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567366" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567366">(Nov 22 2020 at 20:05)</a>:</h4>
<p>I think this sort of thing has little in common with other kinds of allocation APIs then, honestly, and the vast majority of uses would need to be <code>AllocRef&lt;State = ()&gt;</code>...</p>



<a name="217567425"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567425" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567425">(Nov 22 2020 at 20:06)</a>:</h4>
<p>ah, you updated...</p>



<a name="217567436"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567436" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567436">(Nov 22 2020 at 20:06)</a>:</h4>
<p>it's mostly just formatting</p>



<a name="217567450"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567450" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567450">(Nov 22 2020 at 20:06)</a>:</h4>
<p>well you added a huge code block</p>



<a name="217567481"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567481" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567481">(Nov 22 2020 at 20:07)</a>:</h4>
<p>Yeah this feels like a different kind of API. I also am unsure you can make AllocRef take &amp;mut self (and have it be meaningfully mutable) without a big redesign... Note the <code>by_ref</code> function</p>



<a name="217567563"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567563" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567563">(Nov 22 2020 at 20:09)</a>:</h4>
<p><span class="user-mention" data-user-id="209168">@Thom Chiovoloni</span> what's the problem with by_ref?</p>



<a name="217567575"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567575" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567575">(Nov 22 2020 at 20:09)</a>:</h4>
<p>i guess another way would be having the allocator not return NonNull&lt;[u8]&gt; but something else?</p>



<a name="217567580"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567580" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567580">(Nov 22 2020 at 20:09)</a>:</h4>
<p>You wouldn't be able to allocate out of a <code>&amp;impl AllocRef</code> anymore.</p>



<a name="217567582"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567582" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567582">(Nov 22 2020 at 20:10)</a>:</h4>
<p>like an associated type?</p>



<a name="217567634"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567634" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567634">(Nov 22 2020 at 20:10)</a>:</h4>
<p>have another trait like AllocBuffer which can be converted to a [u8] slice</p>



<a name="217567636"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567636" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567636">(Nov 22 2020 at 20:10)</a>:</h4>
<p>that might be a better api actually</p>



<a name="217567822"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217567822" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217567822">(Nov 22 2020 at 20:15)</a>:</h4>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">trait</span><span class="w"> </span><span class="n">AllocRefBuffer</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">fn</span> <span class="nf">as_slice</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nc">NonNull</span><span class="o">&lt;&amp;</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="p">;</span><span class="w"></span>
<span class="w">   </span><span class="k">fn</span> <span class="nf">as_mut_slice</span><span class="p">(</span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nc">NonNull</span><span class="o">&lt;&amp;</span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="p">;</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">struct</span> <span class="nc">Buffer</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">inner</span>: <span class="nc">wgpu</span>::<span class="n">Buffer</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">impl</span><span class="w"> </span><span class="n">AllocRefBuffer</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">Buffer</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">fn</span> <span class="nf">as_slice</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nc">NonNull</span><span class="o">&lt;&amp;</span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w">  </span><span class="o">..</span><span class="p">.</span><span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="w">    </span><span class="k">fn</span> <span class="nf">as_slice_mut</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nc">NonNull</span><span class="o">&lt;&amp;</span><span class="k">mut</span><span class="w"> </span><span class="p">[</span><span class="kt">u8</span><span class="p">]</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="o">..</span><span class="p">.</span><span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">struct</span> <span class="nc">WGPUAllocator</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="n">device</span>: <span class="nc">wgpu</span>::<span class="n">Device</span><span class="p">,</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>

<span class="k">impl</span><span class="w"> </span><span class="n">AllocRef</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">WGPUAllocator</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">type</span> <span class="nc">Buffer</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Buffer</span><span class="p">;</span><span class="w"></span>
<span class="w">    </span><span class="k">fn</span> <span class="nf">alloc</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">layout</span>: <span class="nc">Layout</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nb">Result</span><span class="o">&lt;</span><span class="bp">Self</span>::<span class="n">Buffer</span><span class="p">,</span><span class="w"> </span><span class="n">AllocError</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="kd">let</span><span class="w"> </span><span class="n">buffer</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">device</span><span class="p">.</span><span class="n">create_buffer</span><span class="p">(</span><span class="o">..</span><span class="p">.);</span><span class="w"></span>
<span class="w">        </span><span class="nb">Ok</span><span class="p">(</span><span class="n">buffer</span><span class="p">)</span><span class="w"></span>

<span class="w">    </span><span class="p">}</span><span class="w"></span>

<span class="w">    </span><span class="k">unsafe</span><span class="w"> </span><span class="k">fn</span> <span class="nf">dealloc</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">buffer</span>: <span class="nc">Self</span>::<span class="n">Buffer</span><span class="p">,</span><span class="w"> </span><span class="n">layout</span>: <span class="nc">Layout</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="o">..</span><span class="p">.</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>



<a name="217568133"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568133" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568133">(Nov 22 2020 at 20:23)</a>:</h4>
<p>I think that this gives the API a lot more flexibility.</p>



<a name="217568451"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568451" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568451">(Nov 22 2020 at 20:31)</a>:</h4>
<p>What are some use cases where code would be meaningfully generic over this and the more common cpu-only style allocator?</p>



<a name="217568526"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568526" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568526">(Nov 22 2020 at 20:32)</a>:</h4>
<p><span class="user-mention" data-user-id="209168">@Thom Chiovoloni</span> a basic gpuvec</p>



<a name="217568531"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568531" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568531">(Nov 22 2020 at 20:33)</a>:</h4>
<p>Right but that wouldn't be generic over both, it would only use this</p>



<a name="217568548"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568548" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568548">(Nov 22 2020 at 20:33)</a>:</h4>
<p>what do you mean generic over both, both what?</p>



<a name="217568551"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568551" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568551">(Nov 22 2020 at 20:33)</a>:</h4>
<p>My question is basically: why should this complexity be added to AllocRef vs being a separate trait.</p>



<a name="217568582"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568582" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568582">(Nov 22 2020 at 20:34)</a>:</h4>
<p>so that i can use the rust stdlib vec with a different allocator an get a gpuvec</p>



<a name="217568618"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568618" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568618">(Nov 22 2020 at 20:34)</a>:</h4>
<p>Sure, but now vector needs to track this extra state, which means it needs a lot of extra complexity</p>



<a name="217568633"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568633" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568633">(Nov 22 2020 at 20:35)</a>:</h4>
<p>I somewhat doubt alloc::vec::Vec will want that complexity</p>



<a name="217568637"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568637" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568637">(Nov 22 2020 at 20:35)</a>:</h4>
<p>Moreover, everything parameterized on allocator needs a bunch of extra complexity from state tracking</p>



<a name="217568711"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568711" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568711">(Nov 22 2020 at 20:37)</a>:</h4>
<p>there is not all that many places in vec where you actually tough the raw vector</p>



<a name="217568717"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568717" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568717">(Nov 22 2020 at 20:37)</a>:</h4>
<p>in the second proposal, the complexity from generics goes away since it's an associated type</p>



<a name="217568799"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568799" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568799">(Nov 22 2020 at 20:39)</a>:</h4>
<p>i think that as_slice and as_mut_slice legit handle like 90% of interactions between vec an rawvec</p>



<a name="217568849"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568849" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568849">(Nov 22 2020 at 20:40)</a>:</h4>
<p><a href="https://doc.rust-lang.org/src/alloc/vec.rs.html#300-303">https://doc.rust-lang.org/src/alloc/vec.rs.html#300-303</a></p>



<a name="217568855"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217568855" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217568855">(Nov 22 2020 at 20:40)</a>:</h4>
<p>ok there is relatively few methods</p>



<a name="217581544"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217581544" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tim Diekmann <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217581544">(Nov 23 2020 at 01:42)</a>:</h4>
<blockquote>
<p>Hey gang, I realize that it might be too late to propose changes to the allocator API</p>
</blockquote>
<p>Until <code>AllocRef</code> is stabilized, we can still change the API (e.g. <a href="https://github.com/rust-lang/rust/issues/79286">#79286</a>). </p>
<hr>
<blockquote>
<div class="codehilite"><pre><span></span><code>fn as_slice(&amp;self) -&gt; NonNull&lt;&amp;[u8]&gt; {  ... }
</code></pre></div>

</blockquote>
<p>What is <code>NonNull&lt;&amp;(mut) [u8]&gt;</code> (pointer to borrowed slice), what does it borrow and what is its lifetime? (you probably want to store the bytes of <code>Buffer</code>, but you get a reference). Did you mean <code>NonNull&lt;[u8]&gt;</code>?</p>
<hr>
<blockquote>
<div class="codehilite"><pre><span></span><code>fn alloc(&amp;self, layout: Layout) -&gt; Result&lt;Self::Buffer, AllocError&gt;
</code></pre></div>

</blockquote>
<p>How would that translate to <code>Vec&lt;T, A&gt;</code>? What the vector will do with <code>Buffer</code>, convert it to <code>NonNull&lt;[u8]&gt;</code>? How the API for <code>Vec</code> would look like, does it need adjustments (i.e.: How do I make the vector to allocate a buffer and return it)? I see, how this proposal is useful, but I can't see, how I would use it behind <code>Vec</code> without changing the current behavior.</p>
<hr>
<p>Generally, I think this proposal does not fit the current API but maybe I'm wrong. We are open for proposals but keep in mind, that the usage should be straight forward (Implementing the trait may be more complex). For a wider audience please consider posting a proposal to <a href="https://github.com/rust-lang/wg-allocators/">rust-lang/wg-allocators</a>. <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span></p>



<a name="217582623"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217582623" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217582623">(Nov 23 2020 at 02:10)</a>:</h4>
<p><span class="user-mention" data-user-id="216785">@Tim Diekmann</span> yeah i did mean that . how would it translate to Vec? Buffer would have the a similar API as RawVec does right now. You would not need to change the current buffer behavior too much. I've implemented a GPUVec and it was pretty straight forward. I can try putting together a more detailed proposal. I do think that the API gets a lot of flexibility out of this.</p>



<a name="217586103"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217586103" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217586103">(Nov 23 2020 at 03:31)</a>:</h4>
<p><span class="user-mention" data-user-id="216785">@Tim Diekmann</span> ok quick question, what i think is a bit of a problem api wise is that ideally i would want to disallow people from calling  functions with the default allocator. like if i use a custom<br>
 allocator, then i want to make sure no method that uses the global allocator is called.</p>



<a name="217586168"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217586168" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217586168">(Nov 23 2020 at 03:32)</a>:</h4>
<p>the only way i can think of achieving this is having the allocator trait have a method fn disables_global_allocator() -&gt; bool<br>
which would be checked with an assert whenever you call one of the methods that use the global allocator</p>



<a name="217586204"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217586204" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lokathor <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217586204">(Nov 23 2020 at 03:33)</a>:</h4>
<p>couldn't you just define your own global allocator impl that panics as soon as it's called?</p>



<a name="217586212"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217586212" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Lokathor <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217586212">(Nov 23 2020 at 03:33)</a>:</h4>
<p>or that always returns null?</p>



<a name="217586460"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217586460" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217586460">(Nov 23 2020 at 03:36)</a>:</h4>
<p>oh wait nvm i misunderstood the code<br>
that's actually not a problem since all the functions</p>



<a name="217591459"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217591459" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217591459">(Nov 23 2020 at 05:46)</a>:</h4>
<p><span class="user-mention" data-user-id="216785">@Tim Diekmann</span> it is somewhat more work than i expected. i'm willing to try to do it however i'm curious what do you think are the odds of this being accepted?</p>



<a name="217658912"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217658912" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217658912">(Nov 23 2020 at 17:38)</a>:</h4>
<p><span class="user-mention" data-user-id="216785">@Tim Diekmann</span> ok, i've inspected things some more and i'm not sure how to handle functions like Box::from_raw_in as they take the direct pointer as opposed to a buffer</p>



<a name="217659469"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217659469" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tim Diekmann <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217659469">(Nov 23 2020 at 17:43)</a>:</h4>
<p><span class="user-mention silent" data-user-id="274304">Adam Nemecek</span> <a href="#narrow/stream/197181-t-libs.2Fwg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API/near/217591459">said</a>:</p>
<blockquote>
<p>it is somewhat more work than i expected. i'm willing to try to do it however i'm curious what do you think are the odds of this being accepted?</p>
</blockquote>
<p>That's hard to tell. Personally, I think it will be hard to support this use case but I think it's worth the effort to make a proposal. If we come up with a good API design, which will support the current and your use case without sacrificing usability and performance, there is a decent chance for the proposal. It does not need to be fully fledged out, but we should have a discrete way on how to use it with the current collections</p>



<a name="217659631"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217659631" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217659631">(Nov 23 2020 at 17:44)</a>:</h4>
<p>i'm not sure how to deal with the i mentioned above without resorting to an enum</p>



<a name="217659681"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217659681" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217659681">(Nov 23 2020 at 17:45)</a>:</h4>
<p>like making some enum that is either a unique or a AllocBuffer</p>



<a name="217659955"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217659955" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tim Diekmann <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217659955">(Nov 23 2020 at 17:47)</a>:</h4>
<p>An enum is not an option as it requires more space and <code>Box&lt;T&gt;</code> is required to have the same size as <code>*T</code>. I'd go with the associated type proposal. As long as we can't come up with a better option, it's still possible to require <code>Box::from_raw_in</code> to have the associated type to be the default (<code>()</code> or <code>u8</code>?).</p>



<a name="217660177"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/197181-t-libs/wg-allocators/topic/adding%20State%20type%20to%20the%20AllocRef%20API/near/217660177" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Adam Nemecek <a href="https://rust-lang.github.io/zulip_archive/stream/197181-t-libs/wg-allocators/topic/adding.20State.20type.20to.20the.20AllocRef.20API.html#217660177">(Nov 23 2020 at 17:48)</a>:</h4>
<p><span class="user-mention" data-user-id="216785">@Tim Diekmann</span> my problem is with turning a pointer into a buffer. buffer to pointer is simple.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>